This invention relates to an electronic watermark data insertion apparatus and an electronic watermark data detection apparatus and, in particular, to an electronic watermark data insertion apparatus for embedding identifiable additional information in digital data such as a picture or the like and an electronic watermark data detection apparatus for detecting the identifiable additional information from the digital data.
With the recent improvements of information processing techniques or communication techniques in these years, various kinds of information in the shape of pictures or voices of multimedia information for use in the Internet, a digital video disk or digital versatile disc (DVD), or a digital broadcasting, can be obtained as digital data. As the digital data can be copied or edited without degrading data, a problem occurs such that copies of digitized picture data can easily be made though the copying of the data is illegal.
As one of techniques for preventing such illegal copying, a digital data cryptographic technique is known. According to the digital data cryptographic technique, enciphered digital picture data may be reproduced only by a reproduction system which is provided with a proper cipher recovering key. However, it is disadvantageously impossible to prevent illegal copying which will be carried out once after a cipher is deciphered.
As another technique for preventing the illegal copying of the digital data, a known technique aims to prevent unlawful use and copies of the digital picture data by embedding particular information such as electronic watermark data in the digital picture data.
The electronic watermark data inserted in the picture data such as digital data are classified into two types, that is, visible electronic watermark data and invisible electronic watermark data.
The visible electronic watermark data are represented by special characters, marks, symbols, or other data inserted in pictures so that a composite picture in which the electronic watermark data are composed can be visibly recognized or seen by an observer. Accordingly, although the composite picture results in degradation of picture quality, there is an advantageous effect to visibly appeal, to the users of the composite picture, prevention of unauthorized appropriations such as illegal copying.
The known technique for embedding the visible electronic watermark data in the digital data is disclosed, for example, in U.S. Pat. No. 5,530,759 issued to Gordon W. Braudaway et al. under the title of “COLOR CORRECT DIGITAL WATERMARKING OF IMAGES.” According to Braudaway et al., in order to prevent chromaticities of an original image from altering at points where an electronic watermark image appears, an electronic watermark data insertion apparatus examines, on combining the electronic watermark image with the original image, pixels of the electronic watermark image, and modifying, for each pixel whose value is not a specified “transparent” value, the corresponding pixel of the original image by changing its brightness but its chromaticities. A scaling value for changing the brightness of the pixel is determined, for example, based on the values of the color components, a random sequence, or the value of a corresponding watermark pixel.
Compared with such visible electronic watermark data, the invisible electronic watermark data is, for example, a peculiar data such as information identifying an author. Compared with the visible electronic watermark data, the invisible electronic watermark data is in common to a point where it is inserted in the original image, but differs in a point where it is inserted in the original image with no degradation of picture quality of the original image. That is, the invisible electronic watermark data inserted in the original data cannot be visibly recognized or seen with respect to a composite image.
Accordingly, by embedding the peculiar data which is able to identify the author as the electronic watermark data into the original image, it is possible to identify the author of the original image by detecting the inserted electronic watermark data even after the illegal copying is carried out. In addition, by embedding information indicative of impermission of copying (copying impermission information) into the original image, it is possible to detect the copying impossible information in a reproduction apparatus and to inform a user of the original image of which copying is not permitted is being copied. Furthermore, by embedding information indicative of a prohibition of copying (copy prohibition information) in the original image, it is possible to restrict copying of the original image to a video tape recorder (VTR) or the like by operating a copying prevention mechanism.
Various techniques for embedding the invisible electronic watermark data in a digital data such as a digital image are already known. For example, a first prior art discloses a technique in which the invisible electronic watermark data is embedded into a least significant bit (LSB) of a pixel data of the digital data that has little adverse effect on image quality of the original image for producing a composite image which is obtained by inserting the invisible electronic watermark data in the original image. However, the first prior art is disadvantageous in that only the watermark data may be easily removed from the composite image with no degradation of the image quality of the original image. For instance, it is possible to remove information corresponding to the LSB of the pixel data by using a low pass filter.
In the manner known in the art, an image compressive processing is, for example, carried out according to a JPEG (Joint Photographic Experts Group) and an MPEG (Moving Picture Experts Group) which are international standard coding systems for compressive coding color static image data and color dynamic image data, respectively. Such image compressive processing reduces an amount of the image data by removing an information amount of a part having little adverse effect on such as image quality from an original image. However, if the image compressive processing is carried out for an image data where the electronic watermark data is embedded in the LSB thereof, the electronic watermark data embedded in this portion will be lost. Accordingly, the first prior art is disadvantageous in that it may be difficult to redetect the electronic watermark data.
On the other hand, a second prior art is disclosed, for example, in Japanese Unexamined Patent Publication (JP-A) Tokkai No. Hei 6-315,131 or JP-A 6-315131 under the title of invention of “INFORMATION EMBEDDING DEVICE AND REPRODUCTION DEVICE.” JP-A 6-315131 discloses a technical idea for detecting, by using image correlation between successive frames in an original image data, a particular area where an image is not degraded if the particular area is replaced by a peripheral area thereof on reproduction, and for easily detecting the particular area at a reproduction side by inserting the invisible electronic watermark data in the particular area. The second prior art will later be described in conjunction with FIG. 1 in more detail.
However, as the electronic watermark data is not embedded in all of frames in the second prior art, it is impossible for the technique in the second prior art to prevent illegal copying on frames in which the electronic watermark data is not embedded. In addition, as the second prior art premises that the successive frames are static images or there are no change in the successive frames, it is impossible to identify the particular area in which the electronic watermark data should be embedded in a case where correlation becomes low between successive frames of a dynamic image having furious motion. Accordingly, the second prior art is disadvantageous in that in some cases the electronic watermark data may not be embedded in the original image.
A third prior art, which is another technique regarding embedding of the invisible electronic watermark data, is disclosed, for example, in Japanese Unexamined Patent Publication (JP-A) Tokkai No. Hei 5-30,466 or JP-A 5-30466 under the title of invention of “VIDEO SIGNAL RECORDER AND SIGNAL RECORDING MEIDUM.” JP-A 5-30466 discloses a technical idea for frequency converting, in an electronic watermark data insertion apparatus, an original video signal into a frequency-converted video signal and for embedding, in the frequency-converted video signal, identification information acting as the electronic watermark data having a frequency less than a frequency band of the frequency-converted video signal. In JP-A 5-30466, an electronic watermark data detection apparatus picks up the original video signal by using a wide-band pass filter and picks up the identification information by using a low pass filter.
In the third prior art, the inserted electronic watermark data is not lost although an image compressive processing is carried out and it is possible to insert the electronic watermark data in any frame. However, the third prior art is disadvantageous in that the electronic watermark data can easily be removed from the original video signal by using the wide-band pass filter because the electronic watermark data is inserted in a portion which is lower than a frequency band of the frequency-converted video signal.
In addition, a proposal is made in a fourth prior art for frequency converting an original image data into a frequency-converted image data and for embedding an electronic watermark data in a large area of frequency components of the frequency-converted image data. The fourth prior art is disclosed, for example, in a journal of Nikkei Electronics, No. 660 (1996.4.22), page 13. An electronic watermark data insertion apparatus to which the fourth prior art is applied embeds the electronic watermark data in a frequency component of the original image data. Therefore, in an image processing such as an image compressive processing, filtering, or the like, it is possible to prevent the electronic watermark data inserted in a part having little adverse effect on image quality from being lost. Furthermore, by using a random number according to normal distribution as the electronic watermark data, it is possible to prevent interference between electronic watermark data in a case, for example, of embedding a plurality of electronic watermark data in the original image data. As a result, it is difficult in the fourth prior art to carry out an unlawful action such as to destroy only the electronic watermark data without significantly affecting all of the image data.
That is, the electronic watermark data insertion apparatus according to the fourth prior art converts an original image into frequency components by using a DCT (Discrete Cosine Transform) which is one of orthogonal transforms, for embedding an electronic watermark data calculated as a random number according to normal distribution in the frequency components to produce embedded frequency components, and carrying out an IDCT (Inverse DCT) on the embedded frequency components to obtain a composite image. More specifically, the electronic watermark data insertion apparatus selects first through n-th data f(1), f(2), . . . , and f(n) which have higher values in a frequency area obtained by converting an original image data by using DCT. In addition, the electronic watermark data insertion apparatus selects first through n-th electronic watermark data from the normal distribution having an average of zero and a variance of one and calculates Equation (1) regarding each variable i (where i=1, 2, . . . , n) as follows:F(i)=f(i)+α|f(i)|×w(i)  (1)where α represent a scaling factor. Finally, the electronic watermark data insertion apparatus replaces f(i) with F(i) as the frequency components obtained for each variable i, uses F(i) as frequency components of the composite image in which the electronic watermark data are inserted, and obtains the composite image by using the IDCT.
In order to detect the electronic watermark data inserted in the original image data in the manner which is described above, an electronic watermark data detection apparatus detects the original image data and known electronic watermark data candidates w(i) as follows. That is, the electronic watermark data detection apparatus converts, by using the DCT, the composite image in which the electronic watermark data are inserted into frequency components which have element values F(1), F(2), . . . , and F(n) embedding the electronic watermark data in a frequency region that correspond to f(1), f(2), and f(n). Subsequently, the electronic watermark data detection apparatus extracts electronic watermark data W(i) from f(i) and F(i) according to Equation (2) as follows:W(i)=(F(i)−f(i))/f(i)  (2)
Subsequently, the electronic watermark data detection apparatus calculates, using an inner product of a vector, a statistical similarity C between a known electronic watermark data candidate vector {right arrow over (w)} and an extracted electronic watermark data vector {right arrow over (W)} according to Equations (3), (4), and (5) as follows:{right arrow over (W)}=(W(1), W(2), . . . , W(n))  (3){right arrow over (w)}=(w(1), w(2), . . . , w(n))  (4)C={right arrow over (W)}×{right arrow over (w)}/(WD×wD)  (5)where WD represents an absolute value of the extracted electronic watermark data vector {right arrow over (W)} and wD represents an absolute value of the known electronic watermark data candidate vector {right arrow over (w)}. As a result, when the statistical similarity C calculated by Equation (5) is not less than a particular value, the electronic watermark data detection apparatus determines that the above-mentioned electronic watermark data candidates are embedded in the composite image data.
In the fourth prior art, if an author or the like owning an original image data prepares or generates a composite image data obtained by embedding electronic watermark data in the original image data, the author or the like can effectively carry out judgement of illegality thereof by detecting the electronic watermark data from a digital image data which may be illegally copied in the manner which is described above.
On the other hand, in the fourth prior art, the original image data and the electronic watermark data candidates w(i) are required to detect the electronic watermark data. Accordingly, the fourth prior art can be advantageously used in a case where the author owning an original image carries out judgement of the illegality by performing detection processing for the electronic watermark data on the image data which might have been copied illegally. However, general users cannot carry out the above-mentioned detection processing for the electronic watermark data because the general users have no original image data.
A fifth prior art includes the technique disclosed in the fourth prior art and the improvements therefor, and relates to an electronic watermark data insertion and detection apparatus in an MPEG system that is capable of detecting electronic watermark data which are already embedded in an original image by general users as well as an author.
The electronic watermark data insertion and detection apparatus in the MPEG system according to the fifth prior art divides the original image into a block consisting of (8×8) pixels (which may later be called a (8×8) pixel block) and carries out embedding and detecting of the electronic watermark data every block which is treated as a unit. An electronic watermark data insertion apparatus carries out, in an MPEG coding processing, a DCT processing on the original image to produce first through n-th frequency components f(1), f(2), . . . , and f(n) of an AC component in a frequency region in the order in they become high, and selects first through n-th electronic watermark data w(1), w(2), . . . , and w(n) from normal distribution having an average of zero and variance of one, where n represents a positive integer which is not less than two. Subsequently, the electronic watermark data insertion apparatus calculates an i-th converted frequency component F(i) according to Equation (6) for each variable i (where i=1, 2, . . . , n) as follows:F(i)=f(i)+α×avg(f(i))×w(i)  (6)where avg(f(i)) represents a partial average of three points close to an i-th frequency component f(i), for example, a partial average obtained by averaging absolute values of f(i−1), f(i), and f(i+1). Thereafter, the electronic watermark data insertion apparatus replaces the i-th frequency components f(i) by the i-th converted frequency component F(i) and carries out a succeeding processing of the MPEG coding processing on the first through the n-th converted frequency components F(1) to F(n) so as to obtain a composite image data in which the first through the n-th electronic watermark data w(1), w(2), . . . , and w(n) are embedded in the original image data.
In order to detect the electronic watermark data embedded in the composite image data, an electronic watermark data detection apparatus detects electronic watermark data candidates w(i) as known data as follows. The original image data is not required for detection. The electronic watermark data detection apparatus carries out, in an MPEG expansive processing, carries out inverse quantization on the composite image data to produce a frequency region of a block having first through n-th frequency components F(1), F(2), . . . , and F(n) in the order from the lowest one to the highest one. Subsequently, the electronic watermark data detection apparatus calculates, as an i-th partial average avg(F(i)), an average value of absolute values in three points close to an i-th frequency component F(i) and calculates an i-th electronic watermark data W(i) according to Equation (7) as follows:W(i)=F(i)/avg(F(i))  (7)
Furthermore, the electronic watermark data detection apparatus calculates an i-th total sum WF(i) of the i-th electronic watermark data W(i) of one image for every variable i. Subsequently, the electronic watermark data detection apparatus calculates, using an inner product of a vector, a statistical similarity C between an electronic watermark data candidate vector {right arrow over (w)} and a total sum vector {right arrow over (WF)} according to Equations (8) through (10) as follows:{right arrow over (WF)}=(WF(1), WF(2), . . . , WF(n))  (8){right arrow over (w)}=(w(1), w(2), . . . , w(n))  (9)C={right arrow over (WF)}×{right arrow over (w)}/(WFD×wD)  (10)where WFD represents an absolute value of the total sum vector {right arrow over (WF)} and wD represent an absolute value of the electronic watermark data candidate vector {right arrow over (w)}. When the statistical similarity C is not less than a particular value, the electronic watermark data detection apparatus determines that the electronic watermark data are embedded in the composite image data.
The electronic watermark data being inserted by the technique as disclosed in the fourth and the fifth prior arts are not removed by a simple filtering processing in the electronic watermark data inserted by the technique disclosed in the third prior art. However, when electronic watermark data are inserted after the DCT processing in the MPEG coding processing, the electronic watermark data according to the fourth and the fifth prior arts may be lost because values of frequency components between a predetermined range are converted into particular quantized values by a quantization processing. Accordingly, in the fourth and the fifth prior arts, a large error occurs between the partial average on insertion and the partial average on detection and it results in lowering a detection precision of the electronic watermark data.
A sixth prior art is disclosed, for example, in Japanese Unexamined Patent Publication (JP-A) Tokkai No. Hei 10-191,330 or JP-A 10-191330 under the title of invention of “SYSTEM FOR INSERTING IDENTIFICATION DATA TO DIGITAL DATA AND DETECTION SYSTEM.” JP-A 10-191330 discloses a technical idea for embedding electronic watermark data after quantizing of an MPEG coding processing and for detecting the electronic watermark data before inverse quantizing of an MPEG expansive processing. With this structure, inasmuch as it is possible to avoid disappearing the electronic watermark data caused by quantization, it is possible to prevent detection precision of the electronic watermark data from lowering caused by an error between partial averages on insertion and on detection.
A seventh prior art, which is still another technique regarding embedding of the invisible electronic watermark data, is disclosed, for example, in Japanese Unexamined Patent Publication (JP-A) Tokkai No. Hei 11-55,639 or JP-A 11-55639 under the title of invention of “ENCODE SYSTEM FOR DIGITAL DATA, ELECTONIC WATERMARK INSERTION METHOD AND STORAGE MEDIUM STORING CONTROL PROGRAM.” JP-A 11-55639 discloses a technical idea for inserting an electronic watermark data into a digital image data comprising a series of field data such as an odd number field and an even number field in NTSC (National Television System Committee) or a Y component (a luminance component), a U component (a first chrominance component), and a V component (a second chrominance component), each electronic watermark data being inserted into the corresponding fields. Accordingly, it is possible to insert a plurality of electronic watermark data in one original image data by a very simple structure.
Now, attention will be directed to a human's visual characteristic where a dynamic image or a moving picture has a higher sensible resolution than a static image or a still picture. It is necessary to weaken electronic watermark data to be embedded in order not to degrade image quality of a composite image in which the electronic watermark data are embedded. In the above-mentioned second through seventh prior arts, signals embedded as invisible electronic watermark data are drastically attenuated by the above-mentioned filtering processing or the above-mentioned image compressive processing. On the other hand, if the embedded signals have large strength, an image quality of a composite image is degraded. In fact the strength of the embedded signals have to be taken into account. This is because the strength of the embedded signals as the electronic watermark data are dependent on a method of image compressive processing or the like. However, in a case where signal strength of the electronic watermark data is not taken into account as the second through the seventh prior arts, it may be impossible to detect the electronic watermark data embedded in the composite image data at a reproducing side. As a result, the second through the seventh prior arts are disadvantageous in that detection precision of the electronic watermark data is considerably deteriorated.
Accordingly, an eighth prior art is disclosed, for example, in Japanese Unexamined Patent Publication (JP-A) Tokkai No. Hei 11-341,452 or JP-A 11-341452 under the title of invention of “DYNAMIC IMAGE ELECTRONIC WATERMARK SYSTEM.” JP-A 11-341452 discloses a technical idea for achieving detection precision at a reproducing side similar by weakly embedding the electronic watermark data in each frame at a range where image quality is not degraded, accumulating the electronic watermark data up to sufficient strength at the reproducing side, and by carrying out judgement for the embedded data.
As described above, in a case where the invisible watermark data are embedded in the original data, the problem is that the invisible watermark data may be removed by a simple filtering processing, an electronic watermark data cannot be inserted in any frame, the electronic watermark data are lost by a quantization processing, and so on.
In addition, as the fourth through the sixth prior arts neither disclose nor teach techniques for embedding a plurality of electronic watermark data in one frame of the composite image, the fourth through the sixth prior arts are disadvantageous in that degradation in the image quality of the composite image occurs in a case where the electronic watermark data are inserted in one frame in common.
Furthermore, although the seventh prior art can embed a plurality of electronic watermark data in a simple structure, the seventh prior art is disadvantageous in that the image quality of the composite image is degraded and detection precision of the embedded electronic watermark data is deteriorated in a case where the strength of the embedded electronic watermark data is not taken into consideration in dependency on a specific characteristic peculiar to the dynamic image.
In contract with those, the eighth prior art includes improvements such that the detection precision by weakly embedding the electronic watermark data in each frame at a range where image quality is not degraded and by accumulating and determining the electronic watermark data up to sufficient strength at the reproducing side.
However, inasmuch as the eighth prior art accumulates and determines signals of the weakly embedded electronic watermark data by using a variable threshold value corresponding to the number of accumulated frames at the reproducing side, the eighth prior art is disadvantageous in that it is impossible to ensure a time interval for which the signals of the embedded electronic watermark data cannot be detected without exceeding the threshold value. More specifically, that the signals of the electronic watermark data are not detected at the reproducing side means that it results in necessity of a very long detection time interval or that it is impossible to quite detect the signals of the embedded electronic watermark data. Accordingly, it may be difficult to analyze a cause where the electronic watermark data are not detected. As a result, it is desirable to suit the detection precision so as to be clear a time interval required for detecting the next inserted electronic watermark data and to embed the electronic watermark data having strength enable to detect at the next time although once does not detect the electronic watermark data. Furthermore, inasmuch as the eighth prior art sets the signal strength of the embedded electronic watermark data in consideration of the detection precision, it is desirable to enable to obtain high fine image quality as far as possible with simple in structure in a case of inserting the electronic watermark data.